Unified Data-Free Compression: Pruning and Quantization without Fine-Tuning

Structured pruning and quantization are promising approaches for reducing the inference time and memory footprint of neural networks. However, most existing methods require the original training dataset to fine-tune the model. This not only brings heavy resource consumption but also is not possible for applications with sensitive or proprietary data due to privacy and security concerns. Therefore, a few data-free methods are proposed to address this problem, but they perform data-free pruning and quantization separately, which does not explore the complementarity of pruning and quantization. In this paper, we propose a novel framework named Unified Data-Free Compression(UDFC), which performs pruning and quantization simultaneously without any data and fine-tuning process. Specifically, UDFC starts with the assumption that the partial information of a damaged(e.g., pruned or quantized) channel can be preserved by a linear combination of other channels, and then derives the reconstruction form from the assumption to restore the information loss due to compression. Finally, we formulate the reconstruction error between the original network and its compressed network, and theoretically deduce the closed-form solution. We evaluate the UDFC on the large-scale image classification task and obtain significant improvements over various network architectures and compression methods. For example, we achieve a 20.54% accuracy improvement on ImageNet dataset compared to SOTA method with 30% pruning ratio and 6-bit quantization on ResNet-34.Comment: ICCV202

Intriguing Findings of Frequency Selection for Image Deblurring

Blur was naturally analyzed in the frequency domain, by estimating the latent sharp image and the blur kernel given a blurry image. Recent progress on image deblurring always designs end-to-end architectures and aims at learning the difference between blurry and sharp image pairs from pixel-level, which inevitably overlooks the importance of blur kernels. This paper reveals an intriguing phenomenon that simply applying ReLU operation on the frequency domain of a blur image followed by inverse Fourier transform, i.e., frequency selection, provides faithful information about the blur pattern (e.g., the blur direction and blur level, implicitly shows the kernel pattern). Based on this observation, we attempt to leverage kernel-level information for image deblurring networks by inserting Fourier transform, ReLU operation, and inverse Fourier transform to the standard ResBlock. 1x1 convolution is further added to let the network modulate flexible thresholds for frequency selection. We term our newly built block as Res FFT-ReLU Block, which takes advantages of both kernel-level and pixel-level features via learning frequency-spatial dual-domain representations. Extensive experiments are conducted to acquire a thorough analysis on the insights of the method. Moreover, after plugging the proposed block into NAFNet, we can achieve 33.85 dB in PSNR on GoPro dataset. Our method noticeably improves backbone architectures without introducing many parameters, while maintaining low computational complexity. Code is available at https://github.com/DeepMed-Lab/DeepRFT-AAAI2023.Comment: AAAI 202

Learning Global-aware Kernel for Image Harmonization

Image harmonization aims to solve the visual inconsistency problem in composited images by adaptively adjusting the foreground pixels with the background as references. Existing methods employ local color transformation or region matching between foreground and background, which neglects powerful proximity prior and independently distinguishes fore-/back-ground as a whole part for harmonization. As a result, they still show a limited performance across varied foreground objects and scenes. To address this issue, we propose a novel Global-aware Kernel Network (GKNet) to harmonize local regions with comprehensive consideration of long-distance background references. Specifically, GKNet includes two parts, \ie, harmony kernel prediction and harmony kernel modulation branches. The former includes a Long-distance Reference Extractor (LRE) to obtain long-distance context and Kernel Prediction Blocks (KPB) to predict multi-level harmony kernels by fusing global information with local features. To achieve this goal, a novel Selective Correlation Fusion (SCF) module is proposed to better select relevant long-distance background references for local harmonization. The latter employs the predicted kernels to harmonize foreground regions with both local and global awareness. Abundant experiments demonstrate the superiority of our method for image harmonization over state-of-the-art methods, \eg, achieving 39.53dB PSNR that surpasses the best counterpart by +0.78dB $\uparrow$; decreasing fMSE/MSE by 11.5\%$\downarrow$/6.7\%$\downarrow$ compared with the SoTA method. Code will be available at \href{https://github.com/XintianShen/GKNet}{here}.Comment: 10 pages, 10 figure

Normative Analysis of Individual Brain Differences Based on a Population MRI-Based Atlas of Cynomolgus Macaques

The developmental trajectory of the primate brain varies substantially with aging across subjects. However, this ubiquitous variability between individuals in brain structure is difficult to quantify and has thus essentially been ignored. Based on a large-scale structural magnetic resonance imaging dataset acquired from 162 cynomolgus macaques, we create a species-specific 3D template atlas of the macaque brain, and deploy normative modeling to characterize individual variations of cortical thickness (CT) and regional gray matter volume (GMV). We observed an overall decrease in total GMV and mean CT, and an increase in white matter volume from juvenile to early adult. Specifically, CT and regional GMV were greater in prefrontal and temporal cortices relative to early unimodal areas. Age-dependent trajectories of thickness and volume for each cortical region revealed an increase in the medial temporal lobe, and decreases in all other regions. A low percentage of highly individualized deviations of CT and GMV were identified (0.0021%, 0.0043%, respectively, P \u3c 0.05, false discovery rate [FDR]-corrected). Our approach provides a natural framework to parse individual neuroanatomical differences for use as a reference standard in macaque brain research, potentially enabling inferences regarding the degree to which behavioral or symptomatic variables map onto brain structure in future disease studies

Modeling Rett Syndrome Using TALEN-Edited MECP2 Mutant Cynomolgus Monkeys

Gene-editing technologies have made it feasible to create nonhuman primate models for human genetic disorders. Here, we report detailed genotypes and phenotypes of TALEN-edited MECP2 mutant cynomolgus monkeys serving as a model for a neurodevelopmental disorder, Rett syndrome (RTT), which is caused by loss-of-function mutations in the human MECP2 gene. Male mutant monkeys were embryonic lethal, reiterating that RTT is a disease of females. Through a battery of behavioral analyses, including primate-unique eye-tracking tests, in combination with brain imaging via MRI, we found a series of physiological, behavioral, and structural abnormalities resembling clinical manifestations of RTT. Moreover, blood transcriptome profiling revealed that mutant monkeys resembled RTT patients in immune gene dysregulation. Taken together, the stark similarity in phenotype and/or endophenotype between monkeys and patients suggested that gene-edited RTT founder monkeys would be of value for disease mechanistic studies as well as development of potential therapeutic interventions for RTT

Functional Differences in the Backward Shifts of CA1 and CA3 Place Fields in Novel and Familiar Environments

Insight into the processing dynamics and other neurophysiological properties of different hippocampal subfields is critically important for understanding hippocampal function. In this study, we compared shifts in the center of mass (COM) of CA3 and CA1 place fields in a familiar and completely novel environment. Place fields in CA1 and CA3 were simultaneously recorded as rats ran along a closed loop track in a familiar room followed by a session in a completely novel room. This process was repeated each day over a 4-day period. CA3 place fields shifted backward (opposite to the direction of motion of the rat) only in novel environments. This backward shift gradually diminished across days, as the novel environment became more familiar with repeated exposures. Conversely, CA1 place fields shifted backward across all days in both familiar and novel environments. Prior studies demonstrated that CA1 place fields on average do not exhibit a backward shift during the first exposure to an environment in which the familiar cues are rearranged into a novel configuration, although CA3 place fields showed a strong backward shift. Under the completely novel conditions of the present study, no dissociation was observed between CA3 and CA1 during the first novel session (although a strong dissociation was observed in the familiar sessions and the later novel sessions). In summary, this is the first study to use simultaneous recordings in CA1 and CA3 to compare place field COM shift and other associated properties in truly novel and familiar environments. This study further demonstrates functional differentiation between CA1 and CA3 as the plasticity of CA1 place fields is affected differently by exposure to a completely novel environment in comparison to an altered, familiar environment, whereas the plasticity of CA3 place fields is affected similarly during both types of environmental novelty

Influenza A Virus Entry Inhibitors Targeting the Hemagglutinin

viruse

Bioavailability of heavy metals bounded to PM2.5 in Xi'an, China: seasonal variation and health risk assessment

Studying the characteristics and health risks of heavy metals in atmospheric fine particulate matter (PM2.5) is a crucial component of understanding atmospheric pollution in China. Integrated 24 h PM2.5 samples were collected in winter and summer 2016 in Xi'an, China. The pollution levels, speciation, and health risks of seven PM2.5-bound metal elements (Al, As, Cd, Cr, Ni, Pb, and Zn) were investigated in this study. The average concentration of PM2.5 was 50.1 +/- 30.4 mu g m(-3) and was higher in winter than in summer. Significant seasonal variations in the elements were also observed. The average concentration ratios of Al, As, Cd, Cr, and Pb decreased in summer by 17.5%, 6.4%, 42.5%, 34.1%, and 61.4% compared with their concentrations in winter, respectively, whereas Ni and Zn increased by 37.7% and 7.6% in summer. The soluble and exchangeable fraction (F1) accounted for large proportions of Cd and Pb concentrations, indicating their greater hazard to the environment and human health. Al, As, and Cr mainly existed in the residual state (F4), which had relatively high stability in particulate matter. Ni was consistently distributed in different forms (F1-F4). The bioavailability evaluation indicated that Pb, Cd, Ni, and Zn were potential bioavailable element which exhibited strong biological toxicity. Although the concentration of Pb was very low, its BI value was the highest. The carcinogenic and non-carcinogenic risks of Cr and As were relatively high, and thus require attention from the government and environmental management departments

Assessment of dental age of children aged 3.5 to 16.9 years using Demirjian's method: a meta-analysis based on 26 studies.

BACKGROUND: A method for assessing dental maturity in different populations was first developed in 1973 by Demirjian and has been widely used and accepted since then. While the accuracy for evaluating dental age using Demirjian's method compared to children's chronological age has been extensively studied in recent years, the results currently available remain controversial and ambiguous. METHODS: A literature search of PubMed, Embase, Web of Science, CNKI and CBM databases was conducted to identify all eligible studies published before July 12th, 2013. Weighted mean difference (WMD) with corresponding 95% confidence interval (95% CI) was used to evaluate the applicability of Demirjian's method for estimating chronological age in children. RESULTS: A meta-analysis was conducted on 26 studies with a total of 11,499 children (5,301 boys and 6,198 girls) aged 3.5 to 16.9 years. Overall, we found that Demirjian's method overestimated dental age by 0.35 (4.2 months) and 0.39 (4.68 months) years in males and females, respectively. A subgroup analysis by age revealed that boys and girls between the ages of 5 to 14 were given a dental age estimate that was significantly more advanced than their chronological age. Differences between underestimated dental ages and actual chronological ages were lower for male and female 15- and 16-year-old subgroups, though a significant difference was found in the 16-year-old subgroup. CONCLUSIONS: Demirjian's method's overestimation of actual chronological tooth age reveals the need for population-specific standards to better estimate the rate of human dental maturation

Association of Habitually Low Intake of Dietary Calcium with Blood Pressure and Hypertension in a Population with Predominantly Plant-Based Diets

This study aimed to assess the association of habitually low dietary calcium intake with blood pressure or hypertensive risk using data from the China Health and Nutrition Survey (CHNS) in 2009. We included 6298 participants (2890 men and 3408 women) aged 18 years or older in this analysis. Food intakes were measured by 3-day 24-h individual recalls combined with a weighing and measuring of household food inventory. The participants were divided into normotensive, pre-hypertensive and hypertensive groups according to their mean blood pressure of three repeated measurements. Six intake levels were decided by percentiles of gender-specific dietary calcium intakes (P0–10, P10–30, P30–50, P50–70, P70–90, and P90–100). Average dietary calcium intakes were 405 mg/day for men and 370 mg/day for women, 80% and 84% of which were derived from plant-based food in men and women, respectively. Multiple linear regression analyses showed that dietary calcium intakes were not related with blood pressure in both genders (all P > 0.05). Logistic regression analyses showed a lower risk of pre-hypertension with higher dietary calcium intakes in women (all Pfor trend < 0.001), but not in men; no association between dietary calcium intake and hypertensive risk was found in both genders (all Pfor trend > 0.05). This study suggests that there are no conclusive associations of habitually low dietary calcium intake with blood pressure or hypertensive risk in Chinese individuals consuming predominantly plant-based diets